Correlogram

First, I change all factor variables (anything with response values that can be considered discrete) to values between -2 and 2, then I correlate all variables in a correlogram which visualizes the cross correlations of each square. The results are pretty much what one would expect, high scores in one area are associated with high scores in another area. The only thing to really notice here is that in general people seem to have a lower likelihood to associate the belief that they will apply what they learned in the next 4 to 6 weeks with most other variables, although it is not that much lower. This could be a lower association in that teachers believe they will be more likely to apply their learning when other variables have lower scores, or the inverse, which seems vastly more likely, especially since most scores are very high on average.

To supplement the results of the correlogram I create a summary statistics table where you can see the number and percentage of each level of agreeing:

Textual Data

I look at the textual data first by simply counting words and seeing what appears most frequently, the first occurrence is NAs which I filter out, along with many other ways to say “no response”. After converting the text columns to one word format and filter out stop words, I group by each kind of text column below.

The most common words for activities that support learning appear to be planning, discussion, and student by a wide margin. This seems pretty reasonable and there isn’t much to read into in that regard.

Initially its obvious that most people don’t bother to fill out the additional comments part, but the second word count chart here for additional comments indicates there might be some timeliness issues or other need for better partitioning.

Again it looks like time is a big factor, lessons and planning show up here a lot too.

A lot of expectable themes show up looking at things to try out, teachers seem to be very interested in “text”, “learning strategy”, and relating it to “students.”

“Collaboration” appears to be working well, and “discussions” appears too so group based stuff seems very successful. “Time” makes another appearance as well.

The most common words for people who bothered to fill out their reason for the rating was that it was “informative”, “helpful”, and “knowledgeable” which are all positive. “Time” and “feel” make appearances at the bottom indicating some potential negative feedback.

Looks like people are doing a pretty good job in general.

Bigram

Let’s take a look all the responses at once, first as words, then as a bigram, we see what we have been seeing this whole time, more time is a big request:

Of course we have to look at the actual data to interpret what more time means - here I quickly sample 30 random comments from the 232 unique entries that have “more time” in them to see what most say. There’s no specific pattern that emerges other than a general request for more time which almost entirely comes under the “What could have improved your experience” question column.

Networking

Now I create a network of the bigrams, I notice there are some weird special characters in the data: ö, ä, ñ appear on the center-right of the Fruchterman-Reingold network, as well as in the upper right. I took a look at the appearances in the data and it seems like some stuff got miscoded on entry in the survey, there’s 655 entries that contain ä, and a lot of them look like this: “It‚Äö√Ñ√¥s boring”, which is pretty difficult to interpret. The network itself establishes an “alpha” (the level of opaqueness) based on how rare or common the bigram is, and directionality is established by the arrows. Intepreting this is more of an art than a science, but my main takeaway here is that working in small groups is very highly valued based on the central node in the network where many different words appear to connect to “group”.

Sentiment Analysis with AFINN

I use the AFINN lexicon for sentiment analysis, which gives a numeric sentiment value for each word, with positive or negative numbers indicating the direction of the sentiment.

I visualize this with a plot that shows words going in the “right” or “wrong” direction based on the sentiment analysis using a bar plot. A lot of the “good” ones here are also not great when taken into context of being preceded by “not”. I filter for the top 20 values of contribution which is the count multiplied by the “value” according to AFINN in this chart:

With these word combinations it is easy to comb through the data and see where negative comments are made. Below I again print out the relevant text containing “not + word” to demonstrate actual responses I am aggregating:

Response 4 in the table jumps out at me as very detailed and considerate, “The discussions by the instructors were very inspiring because they were genuinely interested in us and dedicated to helping us see the philosophy of this curriculum. However my answer to the question of recommending this course is low because of the covid situation in our world. My answer is not because of the integrity of the course, the suggestions or the”gist" of the course. The instructors were excellent. The philosophy of equity and responsibility is also. But I cannot recommend this course in light of covid-19. I feel teaching EL through distance learning with integrity would not be possible." This response mirrors the few low scores I did see in the likeliness to recommend to a colleague or friend earlier, covid has made it much more difficult to make this style of course worthwhile.

On a positive note all the text relating to “not alone” is great feedback, with 6 people mentioning under “what went well in the session” that they felt they were not alone. Once again I am utterly confused by responses like 38 where special characters seem to have randomly replaced text, it is still legible but quite strange. There’s a lot to pull from actually reading comments here, so I’ll just mention one more - in responses 15, 18, 19, and 20 all mention the convention center setup as poor, and the chairs as uncomfortable, which is likely outside of anyone at Teaching Lab’s control but it seems like a non-ideal setting.

Word-topic Probabilities

The tidytext package provides a function to extract the per-topic-per-word probabilities called \(\beta\) from a LDA_VEM topic model with a number of topics proportional to the response types. Looking at the response probabilities per topic it looks like more time continues to be the biggest theme, and group work again emerges as a success story for best learning activities. “Nothing”, has the highest probability response to what could have improved your experience, which is a really good sign!